Learning Object Models on a Robot using Visual Context and Appearance Cues

Visual object recognition is an important challenge to widespread deployment of mobile robots in real-world domains characterized by partial observability and unforeseen dynamic changes. This paper describes an algorithm that enables robots to use motion cues to identify (and focus on) a set of interesting objects, automatically extracting appearance-based and contextual cues from a small number of images to efficiently learn representative models of these objects. Object models learned from relevant image regions consist of: (a) relative spatial arrangement of gradient features; (b) graph-based models of neighborhoods of gradient features; (c) parts-based models of image segments; (d) color distribution statistics; and (e) probabilistic models of local context. An energy minimization algorithm and a generative model of information fusion use the learned models to reliably and efficiently recognize these objects in novel scenes. All algorithms are evaluated on wheeled robots in indoor and outdoor domains.